Dynamics of Batch Learning in Multilayer
نویسنده
چکیده
This paper investigates the dynamics of batch learning in multilayer neural networks. First, we present experimental results on the behavior in the steepest descent learning of multilayer perceptrons and linear neural networks. From the results of both models, we see that strong overtraining, the increase of generalization error, occurs in overrealizable cases where the target function is realized by a smaller number of hidden units than the model. Next, under the assumption of asymptotical limit, we mathematically prove the existence of overtraining in overrealizable cases of linear neural networks. From this theoretical analysis, we know that the overtraining is not a feature observed in the nal stage of learning , but it occurs in the intermediate interval of time and forms the global shape of a learning curve.
منابع مشابه
Dynamics of Batch Learning in Multilayer Neural Networks
We discuss the dynamics of batch learning of multilayer neural networks in the asymptotic limit, where the number of trining data is much larger than the number of parameters, emphasizing on the parameterization redundancy in overrealizable cases. In addition to showing experimental results on overtraining in multilayer perceptrons and three-layer linear neural networks, we theoretically prove ...
متن کاملEffect of Batch Learning in Multilayerneural
This paper discusses batch gradient descent learning in mul-tilayer networks with a large number of statistical training data. We emphasize on the diierence between regular cases, where the prepared model has the same size as the true function , and overrealizable cases, where the model has surplus hidden units to realize the true function. First, experimental study on multilayer perceptrons an...
متن کاملDynamics of Batch Learning in Multilayer Networks { Overrealizability and Overtraining {
This paper investigates the dynamics of batch learning of multilayer neural networks in the asymptotic case where the number of training data is much larger than the number of parameters. We consider regression problems assuming noisy output data. First, we present experimental results on the behavior in the steepest descent learning of multilayer per-ceptrons and three-layer linear neural netw...
متن کاملReinforcement learning for robot soccer
Batch reinforcement learning methods provide a powerful framework for learning efficiently and effectively in autonomous robots. The paper reviews some recent work of the authors aiming at the successful application of reinforcement learning in a challenging and complex domain. It discusses several variants of the general batch learning framework, particularly tailored to the use of multilayer ...
متن کاملLearning in Neural Networks and an Integrable System
This paper investigates the dynamics of batch learning of multilayer neural networks in the asymptotic case where the number of training data is much larger than the number of parameters. First, we present experimental results on the behavior in the steepest descent learning of multilayer perceptrons and three-layer linear neural networks. We see in these results that strong overtraining, which...
متن کامل